1. Field of the Invention
This invention relates to the field of video surveillance, and in particular to a system and method for tracking objects that traverse the fields of view of multiple cameras.
2. Description of Related Art
Video surveillance systems often include object-tracking capabilities, wherein an identified object, or target, is continually identified as the object travels within a scene, or set of scenes. Generally, multiple objects are simultaneously tracked. The objects may include people, vehicles, parcels, and so on.
In a typical surveillance system, multiple cameras are strategically placed to optimize the video coverage provided. As an object traverses a camera's field of view, an object-tracking module searches each sequential image frame for the target, based on the appearance of the target within a prior image frame. The appearance of the target in each sequential frame will not generally be identical to its appearance in the prior frame, due to changes in the object's posture and orientation as it moves, or due to changes in lighting conditions, camera angle, as the object enters different areas, and so on. Generally, the amount of change from frame to frame is limited, and the identification of a similar, albeit not identical, object in each sequential frame is fairly straightforward.
Object-tracking becomes more difficult if a target disappears from a scene, then reappears in a different scene. Such a disappearance-then-reappearance can occur, for example, as the target traverses from one camera's field of view to another camera's field of view, as the target exits through a door from one surveillance area to another, as the target enters then exits a non-surveilled area, such as a rest-room, closet, or stairwell, and so on. The difficulty arises from the fact that the target's reappearance differs in time and/or space from when and/or where it disappeared. The lighting conditions may differ on each side of a doorway, the target may change appearance while out of sight, the camera angles may differ, a different object may appear in the other scene, while the target is still out of sight, and so on. The difficulty is further compounded because some objects may disappear and not re-appear in a scene, for example when a target leaves the surveilled area, or areas, entirely.
A variety of techniques have been developed to facilitate the tracking of objects as they enter and exit the fields of view of multiple cameras. Generally, these techniques include a prediction of which camera's field of view will next contain the target, so that the tracking function can be “handed-off” to the images from this next camera. Many of these techniques rely upon an overlap of the fields of view, so that the principles of conventional frame-to-frame image tracking algorithms can be applied. Other techniques rely upon a mapping of the fields of view to a model of the physical environment, and a corresponding mapping of a target's trajectory within this physical environment. U.S. Pat. No. 6,359,647, “AUTOMATED CAMERA HANDOFF SYSTEM FOR FIGURE TRACKING IN A MULTIPLE CAMERA SYSTEM”, issued 19 Mar. 2002 to Soumitra Sengupta, Damian Lyons, Thomas Murphy, and Daniel Reese, discloses a system for automatic camera handoff and is incorporated by reference herein. The approximate physical location of an object is determined from the displayed image using object tracking. The system determines which cameras' potential fields of view contain the object by determining whether the object's determined physical location lies within the bounds of each camera's field of view. When the object is at the bounds of the selected camera's field of view, the system automatically selects another camera and communicates the appropriate information to the figure tracking process to continue the tracking of the figure using this other camera. To properly map the object's trajectory to the physical environment based on images of the object from a camera generally requires a “calibration” of the camera to the physical environment.
In “BAYESIAN MULTI-CAMERA SURVEILLANCE”, Proceedings of Computer Vision and Pattern Recognition, 1999 II:253–259, V. Kettnaker and R. Zabih describe an object-tracking system wherein a “chain of appearances” is created for a tracked object as the object traverses the fields of view of multiple cameras. In addition to using the visual data, the system uses a stochastic model that mathematically expressed how likely it was that an object last seen in one camera would be observed next in a different camera at a particular time. To facilitate the tracking of multiple objects, the image information from all of the cameras is stored in a common database, and various possible object paths are postulated. Using a Bayesian model, the optimal solution is the set of object paths with the highest posterior probability, given the observed data. The maximum a posteriori solution is approximated using linear programming techniques.